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Chang,  Connie  K.N.  “A  New  Lexicon  and  Framework  for  Analyzing  the  Internal 
Structures  of  the  U.S.  ATP  and  Its  Analogues  Around  the  World.”  The  Journal  of 
Technology  Transfer,  vol.  23(2):67-73,  Summer  1998.  This  paper  constructs  a lexicon 
and  an  analytical  framework  with  which  to  understand  and  make  comparisons  between 
various  national  technology  programs.  It  identifies  and  discusses  a representative  sample 
of  important  program  design  features,  e.g,  eligibility  requirements,  the  nature  of  the 
research,  technical  scope,  the  selection  process,  and  public-private  financial 
arrangements,  and  provides  examples  of  national  programs  embodying  alternative 
design  choices. 

Key  search  words:  Advanced  Technology  Program,  foreign  technology  programs, 
national  technology  programs,  program  design,  research,  technology  policy. 

, “ATP  Eligibility  Criteria  for  U.S.  Subsidiaries  of  Foreign-owned 

Companies:  Legislation,  Implementation,  and  Results.”  NISTIR  6099  - January  1998. 
This  report  addresses  the  criteria  set  forth  by  Congress  that  U.S.  subsidiaries  of 
foreign-owned  companies  must  meet  as  a condition  of  receiving  ATP  funding.  It  provides 
information  and  statistics  on  foreign  eligibility  and  participation  and  can  be  a useful 
resource  for  potential  applicants. 

Key  search  words:  Advanced  Technology  Program,  eligibility  criteria,  foreign 
participation,  technology  policy. 

CONSAD  Research  Corporation.  “Advanced  Technology  Program  Case  Study:  The 
Development  of  Advanced  Technologies  and  Systems  for  Controlling  Dimensional 
Variation  in  Automobile  Body  Manufacturing.”  OCR  97-709  - March  1997.  This  study  of 
the  ''2mm  project  "first  investigates  the  effects  of  the  ATP -sponsored  project  at  the  firm 
level  to  determine  the  technological  changes  that  resulted  from  the  project;  the  role 
played  by  the  ATP;  and  the  nature  of  the  direct  economic  impacts  that  are  expected  to 
result  from  the  technological  changes.  The  information  obtained  in  the  background 
investigation  is  then  used  to  estimate  the  national  economic  impacts  of  the  project, 
omitting  at  this  time  possible  impacts  from  future  adoption  of  the  technology  by  user 
industries  other  than  the  automobile  industry. 

Key  search  words:  Advanced  Technology  Program,  automobile  industry,  case  study, 
economic  analysis,  economic  evaluation,  economic  methodology,  technology  policy. 

Jaffe,  Adam  B.  “Economic  Analysis  of  Research  Spillovers:  Implications  for  the 
Advanced  Technology  Program.”  OCR  97-708  - December  1996.  This  paper  by  Jaffe 
identifies  three  forms  of  spillovers  that  are  of  particular  importance  to  the  ATP; 
knowledge  spillovers,  market  spillovers,  and  network  spillovers.  ATP’s  task  of  funding 
high-social-return  projects  that  would  otherwise  have  been  under  funded  will  be 
facilitated  by  considering  those  characteristics  that  contribute  to  spillovers.  This  paper 
identifies  particular  characteristics  of  the  market  and  technological  environment  that 
make  spillovers  more  or  less  likely. 
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Key  search  words:  economic  analysis,  economic  evaluation,  externalities,  market  failure, 
private  and  social  costs  and  benefits,  r&d  spillovers,  social  returns,  technology  diffusion, 
technology  policy. 

Kelley,  Maryellen  R.  “From  Mission  to  Commercial  Orientation:  Perils  and  Possibilities 
for  Federal  Industrial  Technology  Policy.”  Economic  Development  Quarterly,  vol.  11, 
No.  4:313-328,  November  1997.  This  paper  by  Kelley  examines  the  new  commercial 
orientation  of  the  U.S.  government’s  post  Cold-War  technology  programs.  The  Advanced 
Technology  Program  initiatives  reviewed  in  this  paper  signify  a change  in  direction  from 
the  strictly  military-led,  mission-driven  approach  of  the  past  towards  a shared  goal  of 
assisting  industry  in  achieving  technical  advances  that  provide  both  private  and  social 
benefits  to  the  nation  as  a whole.  This  paper  concludes  that  what  lies  ahead  of  a new 
post-cold  war  technology  policy  is  likely  to  follow  an  evolutionary  path  and  a redefinition 
of  the  roles  and  relationships  between  government  and  industry. 

Key  search  words:  advanced  technology  program,  economics  of  science  & technology, 
government-industry  relationships,  private  and  social  costs  and  benefits,  technology 
policy. 

Kelley,  Maryellen  R.,  and  Ashish  Arora.  “The  Role  of  Institution-Building  in  U.S. 
Industrial  Modernization  Programs.”  Research  Policy,  25:265-279,  (1996).  In  this  paper 
Kelley  and  Arora  evaluate  over  20  different  U.S.  industrial  modernization  programs  and 
their  effectiveness  in  addressing  issues  of  technology  adoption.  They  identify  the 
“institution-building”  approach  which  emphasize  inter -firm  learning.  The  government 
serves  as  the  catalyst  for  bringing  technology  developers  together  with  potential  users 
and  for  providing  closer  linkages  between  organizations  that  are  the  technology  leaders 
and  those  that  tend  to  be  followers.  Kelley  and  Arora’ s analysis  suggests  that  those 
programs  which  support  inter-firm  learning  opportunities  stand  a better  chance  of 
contending  with  the  underlying  problems  retarding  technology  transfer,  are  more  likely 
to  gain  acceptability  by  both  technology  leaders  and  followers,  and  are  more  likely  to 
garner  funds  from  the  private  sector. 

Key  search  words:  economic  analysis,  institutional  structure.  Manufacturing  Extension 
Partnership,  private  and  social  costs  and  benefits,  organizational  linkages,  technology 
diffusion,  technology  policy. 

Laidlaw,  Frances  Jean.  “Acceleration  of  Technology  Development  by  the  Advanced 
Technology  Program:  The  Experience  of  28  Projects  Funded  in  1991.”  NISTIR  6047  - 
October  23,  1997.  This  study  demonstrates  that  the  ATP  is  helping  to  overcome  two  types 
of  economic  efficiency  problems  related  to  speed  to  market:  (1)  the  difficulty  of  obtaining 
funding  at  all  to  undertake  long-run,  high-risk,  enabling  technology  development;  and  (2) 
the  difficulty  of  implementing  coordinated,  collaborative  R&D  management  practices 
needed  to  speed  the  conduct  of  R&D  and  the  commercialization  of  the  resulting 
technology. 
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Key  search  words:  acceleration  of  innovation,  Advanced  Technology  Program,  case 
study,  economic  efficiency,  R&D  management  practices,  speed  to  market,  technology 
policy. 

Link,  Albert  N.  “The  Advanced  Technology  Program  Case  Study:  Early  Stage  Impacts 
of  the  Printed  Wiring  Board  Joint  Venture,  Assessed  at  Project  End.”  NIST  OCR  97-722 
-November  1997.  This  report  by  Link  summarizes  numerous  technical  accomplishments 
of  the  printed  wiring  board  research  joint  venture  funded  by  the  ATP  in  1991,  and 
updates  a 1993  study  by  presenting  selected  quantitative  and  qualitative  measures  of  cost 
savings  and  early  economic  impacts  of  the  project.  Although  the  focus  of  the  study  is  on 
cost  savings  from  research  efficiencies,  it  also  documents  early  productivity  gains  from 
the  resulting  new  technical  capabilities. 

Key  search  words:  Advanced  Technology  Program,  case  study,  economic  analysis, 
economic  evaluation,  economic  efficiency,  economic  methodology,  electronics  industry, 
printed  wiring  boards,  R&D  consortia,  research  joint  ventures,  technology  policy. 

Mansfield,  Edwin.  “Estimating  Social  and  Private  Returns  from  Innovations  Based  on 
the  Advanced  Technology  Program:  Problems  and  Opportunities.”  Unpublished  report. 
January  1996.  The  eminent  economist  Edwin  Mansfield  prepared  this  report  in  which  he 
sought  to  further  his  understanding  of  the  ATP  and  the  projects  it  has  funded:  to  assess 
the  willingness  of  company  award  recipients  to  cooperate  directly  with  him  in  proposed 
future  work;  to  determine  the  availability  of  data;  and  to  consider  the  applicability  of  his 
previous  case  study  methodology  to  ATP  case  studies. 

Key  search  words:  case  study,  economic  analysis,  economic  evaluation,  economic 
methodology,  economics  of  science  & technology,  private  and  social  costs  and  benefits, 
R&D  technology  policy,  u.s.  government  technology  policy. 

Powell,  Jeanne.  “Small-Firm  Experience  in  the  Advanced  Technology  Program.” 
Unpublished  report.  June  1998.  This  paper  assesses  plans  and  progress  of  high 
technology  small  firms  funded  by  the  Advanced  Technology  Program  (ATP).  Using  1997 
data  collected  through  the  ATP’s  Business  Reporting  System  (BRS),  the  paper  examines 
project  goals  and  expected  commercial  advantage,  strategies  for  commercialization, 
collaboration  experiences  on  ATP-funded  technologies  for  small  firms  funded  by  the  ATP 
during  the  period  1993  through  1996.  Experiences  for  small  firms  are  compared  with 
experiences  of  the  population  of  all  organizations  funded  by  the  ATP  during  the  same 
period. 

Key  search  words:  commercialization,  economic  analysis,  economic  evaluation, 
economic  methodology,  economics  of  science  & technology,  R&D  technology  policy, 
small  firms,  statistical  analysis,  u.s.  government  technology  policy. 
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. “Development,  Commercialization,  and  Diffusion  of  Enabling  Technologies: 

Progress  Report  for  Projects  Funded  1993-1995.”  NISTIR  6098  - December  1997.  An 
important  component  of  ATP’ s economic  evaluation  plan  for  tracking  project  progress 
and  outcomes  is  the  administration  of  an  electronic  survey  on  a regular  basis  to  all 
project  participants  funded  since  1993.  The  data  from  these  surveys  are  compiled  in  the 
Business  Reporting  System  (BRS)  database.  This  report  by  Powell  summarizes  BRS  data 
covering  periods  through  December  31,  1995,  filed  by  480  companies  in  210  ATP 
projects,  funded  in  19  competitions  during  FY 1993-95.  The  report  provides  an  overview 
of  pathways  to  achieving  targeted  commercial  and  broader  economic  goals;  a status 
report  of  completed  R&D;  an  analysis  of  commercialization  and  technology  diffusion 
activities;  and  analyses  of  other  effects  of  ATP  funding,  such  as  stimulation  of  R&D 
collaborations,  and  increased  private  sector  investment  in  high  risk  and  enabling  R&D. 

Key  search  words:  case  study,  commercialization,  economic  analysis,  economic 
evaluation,  economic  methodology,  economics  of  science  & technology,  R&D 
technology  policy,  small  firms,  statistical  analysis,  technology  diffusion,  u.s.  government 
technology  policy. 

.’’The  ATP’s  Business  Reporting  System:  A Tool  for  Economic 

Evaluation."  The  Evolution  of  Firms  and  Industries.  Helsinki,  Finland:  Tilastokesus 
Statistikcentralen  Statistics  Finland  (1997).  Unpublished  report.  June  1996.  This  report 
by  Powell  describes  on  electronic  survey  instrument  used  for  tracking  the  evolution  of 
projects  towards  achieving  their  business  and  economic  goals,  part  of  the  core  of  ATP's 
program  evaluation  framework.  This  comprehensive  Business  Reporting  System  (BRS) 
consists  of  several  parts.  At  the  beginning  of  the  project,  project  participants  report  on 
their  planned  application  areas  for  the  technology  and  strategies  for  commercialization. 
Annually  they  report  on  progress  towards  implementing  their  commercialization 
strategies  and  on  short-term  economic  impacts  of  the  projects,  including  early  sales 
revenues,  shortened  R&D  cycles,  collaboration  effects,  intellectual  property  creation, 
and  early  job  creation.  Additional  sections  of  the  BRS  now  under  development  will  focus 
on  the  post-ATP  funding  period,  capturing  technology  commercialization  and  diffusion. 
Over  the  following  six-year  period,  participants  will  report  three  times,  increasing  the 
emphasis  on  economic  impacts  of  the  ATP -funded  technology  to  the  nation. 

For  the  reporting  period  ending  December  31,  1995,  there  were  nearly  400  organizations 
in  the  BRS.  Although  most  of  the  participants/projects  reporting  to  date  are  still  in  the 
early  R&D  phases,  adequate  data  is  available  to:  1)  illustrate  some  of  the  types  of 
analyses  possible,  2)  provide  a snapshot  of  commercial  opportunities  that  may  be 
expected  to  result  from  the  awards  portfolio  and  an  approximate  time  line,  3)  give 
evidence  that  companies  are  taking  necessary  steps  for  successful  future 
commercialization,  4)  provide  early  indication  that  the  non-proprietary  information 
developed  with  ATP  funding  is  contributing  to  a shared  knowledge  base,  and  5)  indicate 
patent  filings  attributable  to  the  research  projects. 
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Key  search  words:  economic  analysis,  economic  evaluation,  economics  of  science  & 
technology,  R&D  technology  policy,  statistical  analysis,  u.s.  government  technology 
policy. 

Research  Triangle  Institute.  “Framework  for  Evaluating  the  Economic  Benefits  to  the 
Nation  of  Medical  Technologies  Funded  by  the  ATP,  with  Preliminary  Application  to 
Tissue  Engineering  Projects  Funded  by  ATP  from  1990  to  1996.”  NIST  GCR  97-737  - 
April  1998.  This  study  by  Research  Triangle  Institute  of  Research  Park,  N.C.,  includes 
model  development  for  the  economic  evaluation  of  medical  technologies,  plus 
preliminary  application  of  the  model  for  the  analysis  of  all  tissue-engineering  projects 
funded  by  the  ATP  from  1990  to  1996.  Social,  private,  and  public  rates  of  return  on 
investment  are  estimated  on  the  basis  of  a single,  near-term  use  of  the  multi-use 
technologies. 

Key  search  words:  Advanced  Technology  Program,  biotechnology,  case  study,  economic 
analysis,  economic  evaluation,  economic  methodology,  externalities,  healthcare 
economics,  medical  technology,  private  and  social  costs  and  benefits,  R&D  spillovers, 
social  rate  of  return,  technology  policy,  tissue  engineering. 

Ruegg,  Rosalie  T.  “The  Advanced  Technology  Program's  Evaluation  Plan  and 
Progress.”  7^  International  Forum  on  Technology  Management.  November  1997.  This 
paper  provides  an  overview  of  evaluation  methods  used  by  the  ATP,  while  reporting 
sample  findings.  It  includes  a discussion  of  the  EAO  data  base  and  provides  a listing  of 
completed  EAO  studies.  This  report  also  includes  a diagrammatic  presentation  of 
'‘Spillovers.  ” 

Key  search  words:  economic  analysis,  economic  evaluation,  economic  methodology, 
economics  of  science  & technology,  externalities,  R&D  spillovers,  R&D  technology 
policy,  technology  diffusion,  u.s.  government  technology  policy. 


. “The  Advanced  Technology  Program,  Its  Evaluation  Plan,  and 

Progress  in  Implementation.”  The  Journal  of  Technology  Transfer,  vol.  23(2):5-10, 
Summer  1998.  This  paper  presents  an  overview  of  the  economic  evaluation  metrics  used 
by  the  ATP  in  undertaking  case  studies.  The  paper  describes  a variety  of  studies,  and 
identifies  the  theoretical  methodological  techniques  and  theoretical  approaches 
employed. 

Key  search  words:  economic  analysis,  economic  evaluation,  economic  methodology, 
economics  of  science  & technology,  externalities,  r&d  spillovers,  r&d  technology  policy, 
technology  diffusion,  u.s.  government  technology  policy. 
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. Provided  an  Introduction  for  and  Edited  “Symposium  on 

Evaluating  a Public-Private  Partnership:  The  Advanced  Technology  Program.”  Journal  of 
Technology  Transfer,  volume  23,  No.  2,  Summer  1998.  The  Introduction  positions  the 
evaluation  unit  of  the  ATP,  the  Economic  Assessment  Office,  within  the  context  of  the 
Government  Performance  and  Results  Act  (GPRA).  The  volume  contains  eight  articles  on 
the  ATP’s  evaluation  efforts  providing  examples  of  case  studies  as  well  as  evaluation 
methodological  development. 

Key  search  words:  Advanced  Technology  Program,  economic  evaluation,  public-private 
partnerships. 


. “Guidelines  for  Economic  Evaluation  of  the  Advanced 

Technology  Program.”  NISTIR  5896  - November  1996.  These  guidelines  are  intended  as 
a resource  for  members  of  the  economic  evaluation  community  who  wish  to  perform 
economic  evaluation  studies  of  the  Advanced  Technology  Program  (ATP).  To  help 
members  of  this  community  frame  the  performance  evaluation  problem  in  the  context  of 
the  ATP,  background  information  is  provided  on  the  program,  on  its  evaluation  efforts 
and  accomplishments  to  date,  as  well  as  its  evaluation  objectives  and  principal  areas  of 
interest.  This  paper  also  discusses  procedures  for  submitting  and  evaluating  proposed 
economic  studies  of  the  ATP.  The  guidelines  in  no  way  constitute  a solicitation  for 
proposals  on  evaluation  or  an  obligation  to  fund  any  proposals  that  might  be  submitted; 
they  are  provided  to  improve  the  quality  and  relevancy  of  evaluation  studies,  and  the 
efficiency  and  effectiveness  of  the  ATP/evaluator  interface. 

Key  search  words:  Advanced  Technology  Program,  benefit-cost  analysis,  economic 
analysis,  economic  evaluation,  economic  methodology,  economics  of  science  & 
technology,  impact  assessment,  R&D,  technology  policy. 

. “Risk  Assessment.”  The  Engineering  Handbook.  (CRC  Press, 

1996).  This  chapter  provides  analysts  and  decision  makers  an  overview  of  using  risk 
assessment  techniques  to  help  make  more  informed  choices  in  the  face  of  risk  It  covers 
the  use  of  expected  value  analysis,  mean-variance  criterion  and  coefficient  of  variation, 
risk-adjusted  discount  rate  technique,  certainty  equivalent  technique,  simulation  analysis, 
and  decision  analysis.  Risk  assessment  is  presented  as  a means  of  providing  decision 
makers  with  information  about  the  risk  exposure  inherent  in  a given  decision,  and  also 
with  information  about  their  risk  attitude. 

Key  search  words:  decision  making,  economic  evaluation,  risk,  risk  attitude,  risk 
exposure. 


. "Economic  Methods."  Handbook  of  Energy  Efficiency.  (CRC 

Press,  1997).  This  chapter  provides  a guide  to  making  economically  efficient  choices 
among  alternatives  in  designing  and  sizing  devices,  components,  systems,  and  structures, 
and  in  determining  if  it  is  economically  efficient  to  invest  in  specific  energy-related 
technologies.  It  explains  the  appropriate  use  of  various  measures  of  economic 
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performance,  including  net  present-value  benefits,  rates  of  return,  and  other  measures.  It 
also  provides  an  overview  of  methods  of  assessing  risks  in  investment  decisions,  and  it 
provides  guidance  on  how  to  structure  a benefit-cost  evaluation,  how  to  take  into  account 
the  opportunity  cost  of  capital,  adjust  for  inflation,  handle  taxes  and  subsidies,  and 
establish  study  periods  for  analysis.  These  methods  and  guidance  can  be  applied  not  only 
to  energy-related  investment  decisions,  but  can  be  used  generally  in  assessing  the 
economic  merit  of  alternative  investments  in  technologies. 

Key  search  words:  benefit-cost  analysis,  economic  evaluation,  economic  measures, 
methods  of  evaluation. 

Servo,  Jenny  C.  Dawnbreaker.  “ATP’s  Commercialization  and  Business  Planning 
Guide  for  the  Post- A ward  Period.”  Unpublished  report.  September  1998.  This  Guide  is 
a combination  text/workbook  created  for  the  purpose  of  increasing  the  likelihood  of 
commercialization  success  particularly  by  small  companies  which  receive  funding 
through  the  ATP.  The  intention  is  that  ATP  awardees  might  use  this  guide  from  the 
moment  they  have  been  notified  of  an  award  through  the  completion  of  their  project  and 
on  into  the  future.  The  guide  contains  a series  of  “activities”  intended  to  create 
“cognitive  dissonance.  ” In  other  words,  the  activities  are  designed  to  assist  the 
technology  entrepreneur  develop  useful  business  habits,  as  well  as  to  part  with 
misconceptions. 

Key  search  words:  Advanced  Technology  Program,  business  plan,  commercialization, 
economic  evaluation,  technology  policy, 

Silber,  Bonnie.  Silber  & Associates.  “Survey  of  Advanced  Technology  Program 
1990-1992  Awardees:  Company  Opinion  About  the  ATP  and  its  Early  Effects.”  NIST 
GCR  97-707  - February  12,  1996.  The  Silber  study  provides  a comprehensive  statistical 
analysis  of  one  critical  dimension  of  the  success  of  the  ATP:  the  assessment  of 
participating  organizations  concerning  their  progress  in  carrying  out  the  R&D  projects. 
The  survey  included  150  questions  and  covered  all  60  ATP  projects  initiated  between 
1990  and  1992,  and  125  projects  participants.  The  results  indicate  that  the  participants 
perceive  the  ATP  as  enabling  and  accelerating  their  progress  not  only  in  the  R&D  phase 
but  also  in  the  application  and  commercialization  of  those  technologies  subsequent  to  the 
completion  of  the  ATP  project. 

Key  search  words:  Advanced  Technology  Program,  case  study,  commercialization, 
evaluation,  statistical  analysis,  survey. 

Spender,  J-C.  “Publicly  Supported  Non-Defense  R&D:  the  USA’s  Advanced 
Technology  Program.”  Science  and  Public  Policy.  February  1997,  pp.  45-52. 
Governments  with  mixed  economies  see  many  reasons  to  intervene  in  their  private 
sector ’s  R&D,  but  the  theoretical  justifications  are  hotly  debated  and  poorly  researched. 
The  most  widely  accepted  notion  is  that  private  R&D  has  substantial  public  payoff  and 
that  it  behooves  government  to  subsidize  activities  which  would  otherwise  fall  victim  to 
failures  in  the  marketplace  for  public  goods.  For  this  reason,  like  many  other 
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industrialized  countries,  the  US.  is  experimenting  with  non-military  technology  R&D 
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